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AESTBACT 

In an effort to identify critical evaluation skills, 
interview irfcrmation is analyzed looking at four types of 
differences: differences between age groups, differences between 
children who did and did not change their attitudes after viewing an 
entertainment program, differences among those who ascribed varying 
degrees of credibility to television content, and differences among 
white, black and Puerto Bican subjects, Disconf ixmed hypotheses 
included the importance of the content decided about, the importance 
in itself of accurate knowledge of the television industry, the type 
of real/pretend decisions children make, and the adjudged accuracy of 
children's decisions* The five critical evaluation skills which are 
tentatively identified are (1) explicit and spontaneous reasoning, 
(2) readiness to compare television content to outside sources of 
information, (3) readiness to refer to industry knowledge in 
reasoning about television content, (4) tendency to find televisipn 
content more fabricated or inaccurate, and (5) less positive 
evaluation of television content. (HBC) 



4 * ** V ************************ **♦ 

* Documents acquired by ERIC include many informal unpublished * 

* materials net available from other sources. EEIC makes every effort * 

* to obtain the best copy available. Nevertheless, items of marginal * 

* reproducibility are often encountered and this affects the quality * 

* of the microfiche and hardcopy reproductions ERIC makes available * 

* via the EBIC Dccumeut Reproduction Service (EDRS) . EDBS is net * 

* responsible for the quality of the original document. Reproductions * 

* supplied by EDRS ar€ the best that can be made from the original* * 



EKLC 



us DEPARTMENT OF HEALTH. 
EDUCATION & WELFARE 
NATIONAL INSTITUTEOF 
EDUCATION 

THIS DOCUMtNT HAS BEEN REPRO- 
DUCED EXACTLY AS RECEIVED FROM 
THE PERSON OR ORGANIZATION 0«lGlN. 
ATING n POINTS OF VIEW OR OPINIONS 
STATED DO NOT NECESSARILY REPRE- 
SENT OFFICIAL NATIONAL INSTITUTE OF 
EDUCATION rOS.TlON OR POLICY 



Factors Which Predict the Credibility 
Ascribed to Television 
September 5, 1976 



by 



■PERMISSION TO nEPHOOUCf: THIS COPY- 
niGHTEO MATERIAL HAS htfN GRANTED BY 

Aimec Dorr Leifar 



Aimee Dorr Leifer 



TO CF^lC AND OBGANIZAriONS OPERATING 
UNDER AGREEMENTS WITH THE NATIONAL IN- 
STITUTE OF EDUCATION FURTHER REPRO- 
DUCTION OUTblDi: THE ERIC SYSTEM ME- 
QUIRES PERMISSION OF THE COPYRIGHT 
OWNER 



FACTORS l>rHICIi PREDICT THE CREDIBILITY 
ASCRIBED TO TELEVISION 



Aimee Dorr Leifer 

Harvard University 

Paper presented at the anirial meeting 
of the American Psychological Association 

Washington, D.C. 

September 5, 1976 



The purpose of our work so far has been to identify attitudes, 
information, attentional processes, or reasoning processo.s which 
seem to lead to more critical evaluation of television content. 
To tentatively identify these things, which I'll call critical 
evaluation skills, we have analyzed interview information looking 
for four types of differences: 

1, Differences between the three age groups in each of 
the two samples of subjects: 

2, Differences betv/een children who did and did not change 
their attitudes after viewing one entertainment program; 

3. Differences between those who ascribed varying degrees 
of credibility to television content in the two samples 
of subjects; and 

4. Differences between white, black and Puerto Rican 

^Suents and adults, 

If we found these differences for any measure, we believed we could 
tentatively identify that measure as a critical evaluation skill. 

I would like to take a few minutes now to describe each of these 
comparisons, including the rationale for it, the data base, and some 
cautions about its validity. The rationale and data base for the 
first source of information ~ age differences — • have been amply 
explained already. So I will only urge caution in interpreting our 
findings because the instruments, coding systems, and analyses for 
the two samples were different — thoup.h complementary and because 
developmental changes nay not relate to differences in the effects 
of television at any one age. 
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Our second source of information was the comparison of children 
who did c d did not change their attitudes. Obviously, we believed 
that the programs we showed the children were meant primarily to 
entertain not to influence. We liked the messages in half the 
programs and wanted our own children to accept them, but still the 
more prudent course for all children would be to reject them. Our 
sample included the 47 children Sherry Graves described at the 
beginning of the symposium. In selecting them, the raw attitude 
change scores of 99 children x^7ere adjusted on the basis of pretest 
scores and then rank ordered. Children from approximately the top 
and bottom octiles were designated as changers and children approx- 
imately in the middle quarter were designated as nonchangers. Then 
the sample of changers and nonchar.gers was matched, plus or minus 
one subject per group, for age, race, sex, and type of television 
program viewed. The legitimacy of our subsequent analyses depends, 
of course, on the validity of our identification of children as 
changers and nonchangers. 

Our third source of information was the comparison of those 
who ascribed differing degrees of credibility to television. For 
each of our two samples, subjects were divided;, on the basis of 
answers to a number of specific questions in the interviews, into 
roughly three equal groups — high, medium, and low average 
credibility. We had assumed that average credibility would change 
with age. We found, however, that the dividing points could be 
the same for all three ages of children and the same for the adoles- 
cents and adults. Our major reservations about this source of 
information stem from this lack of age differences and from the 
fact that the average credibility score did not relate to 
children's statuses as changers and nonchangers. 

Our fourt^" and final sourc:; of information was the comparison 
of the white, black, and Puerto Rican subjects in the adolescent 
and adult sample. Children wercj excluded because the changer- 
nonchanger sample included too few blacks. We argued that members 
of American i^inority groups are m.ore likely than whites to be 
skeptical because television does not as often represent life as 
they know it. Blacks and Puerto Ricans were chosen to represent 
American minorities because they are numerically large both in the 
United States and in the Boston area where ve worked. Despite our 
efforts, the Puerto Rican sample was distinctly different from the 
other two. Many were relatively new arrivals from Puerto Rico, 
spoke little English, and were of lover socioeconomic status, l^ile 
this may be an accurate reflection of the Puerto Rican population 
in the Boston area, it also means that television viewing is a 
substantially different experience for them — and our data reflect 
this. So v;e will look mainly at black-white differences here. 

Now that I have described our four sources of information let 
me tell you what do not seem to be critical evaluation skills. 
All four types of measures I will discuss now are ones we thought 
would be critical evaluation skills , but did not turn out that way 
in our data. 
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The first dlsconf irmed hypothesis was the importance of the 
content kindergarteners through adults decided about. As Sherry 
Graves has shown ^ there are clear age changes in what people attend 
to, but these differences do not show up in the rt.her three types 
of comparisons. These differences will guide the selection of 
content to use as a vehicle in teaching critical evaluation skills, 
but in themselves they do not seem to be important as skills. 

The second disconfirmed hypothesis was the importance in itself 
of accurate knowledge of the television industry. As Erin Pr Ins 
demonstrated, there are clear ir^reases from kindergarten to 
adulthood in people's industry knowledge. Similar differences 
do not, however, appear in the other three types of comparisons. 
Thus, we have little reason — other than the age changes — to 
believe that accurate knowledge of the industry x^ould xead to more 
critical use of television. As Erin noted, all of our subjects 
could learn much more about the industry, and, as I will indicate 
later, application of industry knowledge to reasoning about tele- 
vision content does seem to be important. Kence, the industry 
knov/ledge analyses will indicate \<hat information to teach children 
to apply in their reasoning, 

The third disconfirmed hypothesis is that the type of real/ 
pretend decisions children made would matter. I^iile measures 
of children's real/pretend decisions may seem to be quite similar 
to the average credibility score, let me assure you they are not. 
The average credibility score was derived from about six very general 
questions about television's believability, while the decisions 
measure is based on the total number of real/pretend decisions 
ma-.ie throughout the int ,rview, with the average number of decisions 
per subject being 41, vie had assumed that the decisions of less 
sophisticated viewers would more often be real — rather than pretend 
or both real and pretend. We did find the predicted age differences, 
which were not significant, and predicted and significant average 
credibility group differences (F=18.10; df=2,38; p < ,001 for real 
decisions and F'-20,32i df='2,38: p<.001 for pretend decisions), 
but similar differences were not found for changers and nonchangers. 
Ue did not have a similar measure for the adolescent and adult sample. 

Finally, we did not find that the adjudged accuracy of children's 
decisions matteriid, and once apain we do not have a similar measure 
for the adolescents and adults. We had assumed that more critical 
viewers would mal:e more accurate judgments about television content — 
at least accurat(i in our eyes. Ue did find cleav increases with age 
in the adjudged accuracy of children's decisions (F=8.755 df=2,4^:. 
p < *001), but similar differences were not found in the other twt() 
analyses. \ 
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Now that we have disposed of some of the things that do not 
appear to be critical evaluation skills ~ even though we thought 
they might be — let me tell you some of the critical evaluation skills 
we believe we have identified. I should note that none of these 
tentatively identified critical evaluation skills is supported by 
significant Fs, chi-squares, or Ouher means tests for all relevant 
comparisons in both samples. Still there seems to be enough support 
to discuss them as possible critical evaluation skills and then 
to test them in more rigorous experiments. 

The five critical evaluation skills we have tentatively iden- 
tified are: 

1. Explicit and spontaneous reasoninj^; 

2. Readiness to compare television content to outside 
sources of information : 

3. Readiness to refer to industry knowledge in reasoning 
about television content ; 

4. Tendc^ncy to find television content more fabricated 
or inaccurate and 

5. Less positive evaluation of television content. 

I will explain the specific ir.easures subsumed under a critical evalua- 
tion skill when I discuss thac skill. In each discussion I will 
label differences as significant v/hen they were significant and 
simply state the direction of differences when they were not. 

The first critical evaluation skill seems to be explicit and 
spontaneous reasoning about the reality of television content 
(see Figure 1). For the children, we found over the entire interview 
that the use of Mental Process reasons like '*I just figured it out" 
decreased as children got older (F=2.37r df=2,41r p-.lO) and sig- 
nificantly as we moved from changers to nonchangers (F=6.62; 
df=l,41; p .05). It decreased too from high to moderate credibility 
groups and then increased sor'2what for the low credibility group. 



Insert Figure 1 about here 



We also found age differences in the percentage of responses 
wliich could be coded as reasons in the entire ir.cerview (F=10.05; 
df==2,41: p<.001), with the percent increasing with age, and the 
percent significantly higher for nonchangers than changers, in the 
section of the interview where they talked about programs they liked 
to wat>ch (F=4.27; df=^l,41- p<.05). The average Credibility group 
comparison also supported this finding. do not have the same 
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measure for the adolescents and adults, but there is one that's similar. 
That is the percent of all coded responses which were reasoning 
processes. This percent increased slightly as subjects pot older, 
somewhat more as they ascribed less credibility to television, and 
very sliglitly as we moved from whites to blacks, 

We also found for the children that ^re were significant age 
differences in the percentage of coded responses which were given 
spontaneously, without interviewer elicitation, with means increasing 
with aoe (F=15.24; df=2,41: pv.OJl). There was, however, a decrease 
as we moved from changers to nonchangers and no support in the 
credibility group comparisons. The data for the adolescents and 
adults are better. The percentage of unelicited responses war? sig- 
nificantly greater for adults ti.an adolescents (Newman -Keu Is Tesu: 
Adult vs 16, p^.05: Adult v.s 13, p<.05). This percentage also in- 
creased as people ascribed less credibility to television. VJhites 
and blacks did not differ. 

Thus, we have some evidence that people who can talk more 
readily and explicitly about their reasoning may be more critical 
consumers of television, Ue assume that this means that they had 
thought and talked more about these issues before we interviewed 
them. 



Insert Figure 2 about here 



The second critical evaluation skill we have tentatively iden- 
tified is the readiness to compare television content to outside 
sources of information (see Figure 2). For the children, we found 
that references to parents as sources for real/pretend decisions 
were nearly significantly higher for nonchangers than changers over 
the entire interviev: (F=3.02: df=-l,41; p<.10) and significantly 
higher in the section of the interview where they talked about the 
program we had shown them (F=6.69, df=l,41; p<.05). In this latter 
section, references to parents also differed significantly i 'r the 
ascribed credibility groups (F=13.82; df=2,38; p<.001), with references 
increasing as credibility decreased. Over age, as Michael Fo*:r.e 
showed, references to parents remained relatively constant. But 
this should not obscure the finding that information from parents 
seems to be useful. For the children, we also found significant 
age differences in referenr-^s to others (F=3.53: df=2,4l; p<.05), 
with references increasing with age, and a slight increase as we 
n:oved from ch/ rs to nonchangers. There was no support for this 
VAaarure in tht .verage credibility group comparison. A similar 
fjiKtlng occurs with the adolescent and adult sample. Here, the 
measure is references to other authorities, including parents. 
There are no age or ethnicity differences in the use of this but 
there are significant ascril^d credibility group differences (F=2.68: 
df=2.,43- pC.05), with use incicasing as credibility decreases. 
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Tlicsrc arc tv;o other nieasuroa In both the cl>lld vind ndolcfjcent 
and adult wampJcH which relate to this findin(», Michael Forte 
described these ns personal experience, which should increaae to 
Hupport the second critical evaluation nkill, and conceivable ex- 
perience, which should decrease. The results for both measures 
are complicated unfortunately. For the adolescents and adults the 
percent of references to personal experience was a U-shaped function 
of age and it increased as people ascribed Ims credibility to 
television, and slightly as we moved from whiten to blacks. The 
percent >f references to conceivable experience decreased slightly 
as people .r,ot older, didn't chan/,e as they ascribed less credibility 
to telcvlL^lon , and was somewlmt lower for blacks than for whites. 

The findings for thc:;c two measures are more complicated for 
tho children. For references to personal experience in the entire 
interview we found a U--shapcu cuvvc with age and mt ascribed 

crddibility group differences (K^5,93; df«2,38; p* with use 

hlijhest for the low credibility p,roup. There was ; ^'^ for 

thir, measure in the changer-noncl;anp,er comparison. references 
to conceivable experience, we found significant age differences 
(F»10.54; df«=2,A3; p<.001), v/ith use increasing, rather than de- 
creasing, with age, significant ascribed credibility group dif- 
ferences (F'^A.Sl, df«2,38; p<.05), with an inverted U-shaped curve, 
and a decrease as we moved from changers to nonchangers. It may 
be that the use of conceivable experience Increases with age, 
it is certainly very frequent in our adolescent and adult data, 
but that it is not as useful a reasoning process as it might be. 
It is clearer in our data that the use of personal experien * 
is Important at all ages. 

Thus, we have fairly consistent evidence that the readiness 
to compare television content explicitly to one's own experience 
and to use sources of infonration outside of television such as 
parents and other authorities may be a critical evaluation skill. 

The third critical evaluation skill which we have tentatively 
identified is the readiness to refer to knowledge of the industry 
in reasoning about television reality (see Figure 3). For the 
adolescer:t3 and adults we found an increase in the use of this 
category as people got older, with the differences between adults 
and each adolescent c^roup sif^nii'icant (p<.05 by i^ewman-Keuls Test), 
an increase and leveling off ns they ascribed less credibility 
to television, and a significrat increase as we moved from whites 
to blacks (t=3.5- df=43" p<.C01 for planner comparison). From our 
measures of understanding industry economics we must conclude that 
it 1 not accurate understanding^ per se which matters j although 
it was such knowledge us2d in reasoning about television content 
which v/as ceded for adolescents and adults. TJhat does seem to 
TT^atter, however, is the readiness to use this knowledge in evaluating 
content • 
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Hone of the children ever referred to industry economics in 
reasoning about television content. The closest measure we have for 
children, which they did use, was rather general statements about 
the fact that entertainment programming is made up. Here we find 
that nonchangers use this category significantly more than do 
changers when talking about the program we showed them (F=6.08; 
df=l,41- P41.05) and that the use of it increases very slightly as 
children ascribed less credibility to television. There was no 
support for this measure in the age comparison; in fact, use de- 
creased v7ith age and age differences were significant (F=3.34; 
df=2,4l: p<.05). 

The fourth critical evaluation skill which we have tentatively 
identified is the tendency to find television content more fabricated 
or inaccurate (see Figure 4). One measure for the children which 
related to this is that television content is made up, which I have 
just discussed. For the adolescents and adults, this same category 
was used less frequently. Still we found nearly significant age 
differences (F=2.47; df=2,43; p<.10)5 with use increasing with age 
when scores were transformed to equalize varisince, but with an in- 
verted U-shape for the raw means x»fl7hich are graphed) and some 
increases as people ascribed less credibility to television. Black- 
white differences were cilso nearly significant (t=1.4; df=43; 
p<.10 for planned comparisons) uhen we transformed scores to equalize 
group variances, with the transformed means showing blacks higher 
than whites but the reverse for the raw means (which are graphed). 



Insert Figure 4 about here 



A second measure which relates to this skill is the extent to 
which people found the portrayals of their own sex and ethnicity 
group to be larp:ely inaccurate. For the adolescents and adults, 
viewing portrayals as accurate decreased significantly with age 
(X =19.16; df=10' p<,05) and as people ascribed less credibility 
to television (X^=37.S8; df=10: p<.001). Blacks and Puerto Ricans 
were also significantly more likely than whites to find their por- 
trayals inaccurate (X^=25.36; df=10: r<.005). V/e couldn't test 
this measure with children becaut^e only sixteen of them answered 
the question «hen it was asked correctly. 

The fifth, and final, critical evaluation skill which we believe 
we have identified is the ger^iral evaluation people have of tele- 
vision (see Figure 5) . Support for this coraes only from the 
adolescents and adults, as we unfortunately neglected to obtain 
relevant data from children. For the adolescents and adults, we 
found increases in negative evaluations of television content as 
people: got older, increases as they ascribed lees credibility to 
television, with the group differences significant (F=4.92; df-2,42:. 



EKLC 



14 



TELEVISION) FABRlCAteX) ^Nh l/\J ^CCOkPtlE 




__j , 1 




14 



p<,05), and some increase as we moved from whites to blacks. 



Insert Figure 5 about here 



Thus we have tentatively ident:ified five critical evaluation 
skills: 

1, Explicit and spontaneous reasoning* 

2, Readiness to compare television content to outside 
sources of information; 

3, Readiness to refer to industry knowledge in reasoning 
about television content; 

4, Tendency to find television content more fabricated 
or inaccurate; and 

5, Less positive evaluation of television content. 

The teaching experiments which Judith Lemon will describe next 
should help us understand the extent to which these skills can 
be learned and do actually make a difference in how one reacts to 
television. 

Let me end by reiteratinp^ that more critical evaluation, if 
it should be learned, should not lead to wholehearted rejection 
of all of television's messages. Rather it should help children 
differentiate betx^;een content which is meant primarily to inform 
and content which is meant primarily to entertain. Further, it 
should help children identify content which accurately represents 
normative or usual behavior and content which conforms to their 
family's values. 
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